Running head: SUSTAINING SCHOOL- WIDE PBIS 


1 


Critical Incidents in Sustaining School-wide Positive Behavioral 
Interventions and Supports 
Theresa E. Andreou, PhD 
University of British Columbia 
Kent McIntosh, PhD 
University of Oregon 
Scott W. Ross, PhD 
Utah State University 
Joshua D. Kahn, MSEd 
University of Oregon 

REFERENCE: Andreou, T. E„ McIntosh, K„ Ross, S. W„ & Kahn, J. D. (2015). Critical 
incidents in the sustainability of school- wide positive behavioral interventions and 
supports. Journal of Special Education, 49, 157-167. 

Author note: This study served as the first author’s doctoral dissertation, while the second author 
was affiliated with the University of British Columbia. The research reported here was supported 
by the Institute of Education Sciences, U.S. Department of Education, through Grant 
R324A120278 to University of Oregon. The opinions expressed are those of the authors and do 
not represent views of the Institute or the U.S. Department of Education. 



Running head: SUSTAINING SCHOOL- WIDE PBIS 


2 


Abstract 

The purpose of this exploratory study was to identify, categorize, and describe practitioners’ 
perspectives regarding factors that help and hinder sustainability of Tier I (universal) systems 
within School-wide Positive Behavioral Interventions and Supports (SWPBIS). Seventeen 
participants involved in sustaining Tier I SWPBIS over several years within a school district 
were interviewed and asked what events affected its long-term implementation through a 
qualitative approach called Critical Incident Technique. Two hundred and twenty-seven critical 
incidents were recorded and sorted into emergent unitary clusters based on content analysis. 
These categories then underwent rigorous reliability and validity checks, including expert 
analysis, inter-rater agreement, and participant feedback. This process yielded 13 categories that 
represent the participants’ experience of sustainability: Continuous Teaching, Positive 
Reinforcement, SWPBIS Team Effectiveness, Staff Ownership, School Administrator 
Involvement, Adaptation, Community of Practice, Use of Data, Involving New Personnel, 

Access to External Expertise, Maintaining Priority, Staff Turnover, and Conflict of Personal 
Beliefs/Mistaken Beliefs. 

Keywords: positive behavior support, sustainability, systems change, qualitative, implementation 
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Critical Incidents in Sustaining School-wide Positive 
Behavioral Interventions and Supports 

Educators today are being challenged to meet social behavior needs in addition to 
academic objectives, in an environment where time and resources are already stretched to 
capacity. Rising dissatisfaction with reactive and punitive approaches to school discipline has 
resulted in a paradigm shift toward more broad-based, preventive, and ecological approaches to 
supporting student behavior needs. School-wide Positive Behavioral Interventions and Supports 
(SWPBIS; Homer et al., 2009) is one example of a framework for supporting student behavior 
that incorporates the principles of applied behavior analysis, contextual validity, systems change, 
inclusive ethics, and stakeholder collaboration to enhance learning and social environments with 
a preventive, systems-level approach. SWPBIS has emerged as an effective, efficient, and 
socially valid approach for improving behavior outcomes and social competencies (Bradshaw, 
Waasdorp, & Leaf, 2012; Flannery, Fenning, Kato, & McIntosh, 2014; Horner et al., 2009). 

The core features of SWPBIS are applied through a three-tiered continuum of service 
delivery to optimize development and support the range of student needs. At Tier I, universally 
effective strategies are provided to all students by all school personnel to promote social 
responsibility. The school develops school-wide expectations, which are brief in number, 
contextually defined, and positively worded. Expectations are then posted, defined using a 
matrix that provides specific examples in each setting, taught explicitly, and reinforced 
strategically. Explicit teaching includes targeted lessons, demonstrations in settings where 
problem behaviors often occur, and practice with performance feedback. Systematic 
reinforcement of positive behaviors involves high rates of descriptive feedback, often 
accompanied by external rewards. For example, students might receive acknowledgment tickets, 
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which can be spent at a school store or to attend a school event. Finally, in Tier I, a continuum 
of instructional consequences for violations of expectations are developed. A formalized process 
of collecting, entering, and analyzing discipline data is used to direct decision making and 
student support. Data are examined regularly to determine patterns, and practices are modified 
to improve effectiveness and efficiency. Students not successful with Tier I supports are 
provided additional supports (Tier II or III supports) as needed (Bambara, Nonnemacher, & 

Kem, 2009). 

Sustainability 

Although SWPBIS has evidence of effectiveness, the sustainability of even the most 
efficacious practices cannot be assumed (Santangelo, 2009). As SWPBIS scales up, educators, 
researchers, and policy makers need to know considerably more about what factors benefit and 
hinder the sustained implementation of SWPBIS. Within education, several key elements of 
sustainability have been highlighted. Studies indicate that practitioner buy-in plays a significant 
role in continuing a practice (Baker, Gersten, Dimino, & Griffiths, 2004). Hieneman and Dunlap 
(2001) found that sustainability was related to its acceptability, contextual fit, and degree of 
individual and agency collaboration. Empirical studies have noted the importance of a team- 
based approach and use of data as powerful predictors of sustainability (Coffey & Horner, 2012; 
McIntosh et al., 2013). Administrator support has also consistently been highlighted as important 
(Kincaid, Childs, Wallace, & Blase, 2007; McIntosh et al., 2014). Finally, district-level support 
has also been found to have a strong effect on sustainability (Santangelo, 2009). However, there 
remains little empirical guidance regarding exactly what steps at the school and district level are 
most likely to maximize sustainability (McIntosh, Filter, Bennett, Ryan, & Sugai, 2010). 


The Promise of Qualitative Research in Understanding Sustainability 
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Research in the field of education has recognized the need for more diverse methods of 
investigation, including the use of qualitative studies, which “shed light on unanticipated 
consequences of current practice and lead to a better conceptualization of interventions and 
reappraisal of practice” (Gersten, Baker, Smith-Johnson, Flojo, & Hagan-Burke, 2004, pp. 330- 
331). Biglan (2004) noted the importance of understanding complex school and district systems 
change processes through studying individual examples before turning to experimental trials. 
Regarding SWPBIS specifically, Singer and Wang (2009) argued that qualitative methodologies 
can complement the more commonly used experimental research methodologies (e.g., single 
case design). The subjective dimensions of the process of sustaining SWPBIS implementation 
can be determined through careful observation and asking people directly to talk about their lived 
experiences (Brantlinger, Jimenez, Klingner, Pugach, & Richardson, 2005). 

In response to this need, we used a qualitative design to evaluate factors perceived as 
helping or hindering the long-term sustainability of Tier I SWPBIS in schools in a district that 
had been implementing SWPBIS for over 15 years. Qualitative research asks questions regarding 
“how” or “why” sustainability occurs under real world conditions, which can help the field more 
fully understand sustainability, as well as what steps can be taken to enhance it. Elias, Zins, 
Graczyk, and Weissberg (2003) encouraged researchers to unearth practitioners’ implicit theories 
of how practices are sustained by documenting and exploring their day-to-day experiences. A 
better understanding of the process by which sustainability occurs can be beneficial in terms of 
expanding the knowledge base, but equally important, it can provide a stronger theoretical 
foundation for specific interventions to enhance program sustainability. 
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The Present Study 

Sustained implementation of educational initiatives is a rare phenomenon (Fixsen, 
Naoom, Blase, Friedman, & Wallace, 2005), and previous qualitative research has focused more 
on failures to sustain practices (Santangelo, 2009; Sindelar, Shearer, Yendol-Hoppey, & Fiebert, 
2006), the current study capitalized on a unique opportunity to gain insight into the grassroots 
factors in sustaining effective practices. We used a qualitative approach to examine school and 
district personnel’s underlying stories of sustainability of Tier I SWPBIS as it unfolds in 
practice. Research questions that were explored in this study included: (1) What incidents help 
the sustainability of SWPBIS? and (2) What incidents hinder the sustainability of SWPBIS? 

Method 

Setting and Participants 

We studied a school district in rurcd British Columbia with schools that had been 
implementing SWPBIS for over 15 years. The district included 32 schools, with an enrollment of 
14,000 students. Specific settings for this study included the district office and three elementary 
schools, which had each been implementing SWPBIS for between 10 and 14 years. At each 
school, implementation of Tier I SWPBIS was documented as adequate (range: 86 - 89%) on the 
School-wide Evaluation Tool (Sugai, Lewis-Palmer, Todd, & Horner, 2001), a research 
validated external evaluation of Tier 1 SWPBIS fidelity of implementation for the year of the 

study. 

Participants were a convenience sample of 17 educators: 4 current and former 
administrators, 4 district consultants, 3 special education teachers, and 6 general education 
teachers familiar with the aim and the daily activities of SWPBIS. Of these, 12 (71%) were 
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female. Minimum criteria for participation included receipt of training or professional 
development in SWPBIS and at least two years of experience implementing Tier I SWPBIS. 

Qualitative Approach 

Critical Incident Technique (CIT; Flanagan, 1954) is a phenomenological qualitative 
research method, emerging from industrial and organizational psychology, that focuses on 
participant lived experiences to draw pragmatic insights concerning real-world problems 
(Butterfield, Borgen, Amundson, & Maglio, 2005; Flanagan, 1954; Woolsey, 1986). Studies 
suggest that it can be both a reliable and valid research tool because of its focus on rich analysis 
of observable behaviors (Andersson & Nilsson 1964; Kain, 2004; Ronan & Fatham, 1974). CIT 
has been used in education to examine principals’ change in support for SWPBIS (McIntosh, 
Kelm, & Canizal Delabra, in press), teacher collaboration (Kain, 1997), and teacher job 
satisfaction (Engelking, 1986). 

CIT is based on identification and analysis of Critical Incidents (CIs), defined as “any 
reported occurrence that could be translated into specific, observable, and behavioral terms” 
(Bedi et al., 2005, p. 314), with a particular focus on observable, measurable events that are 
perceived as helping or hindering a process of interest (e.g., regular team meetings). This focus 
on observable, measurable events mitigates concerns regarding the accuracy of participant recall 
(Kain, 2004). Moreover, individual interviews and follow-ups with multiple respondents are 
used to compile and analyze a set of CIs, which allows researchers to identify potential 
inconsistencies in recall and perform member checks to clarify accounts (Kain, 2004). 


Procedure 



Running head: SUSTAINING SCHOOL- WIDE PBIS 


8 


Recruitment. Schools and participants were recruited by the district SWPBIS 
coordinator based on his experience in implementing SWPBIS in the district. Informed consent 
was obtained first from the principals of each school, then all participants at the interview. 

Interview process. A semi-structured interview questionnaire was adapted from Bedi et 
al. (2005) and Butterfield et al. (2009). To ensure fidelity to the methodology, the interview 
protocol was reviewed by independent researchers with specialized expertise in the areas of 
methodology (CIT interviewing) and content (SWPBIS), as recommended by Bedi et al. (2005). 

Data were collected through one face-to-face interview conducted by the first author with 
each individual participant. All interviews were conducted over a two-month period by the first 
author. With the permission of the participants, the interviews were tape recorded and 
transcribed to facilitate thorough and reliable discourse analysis (Butterfield et al., 2005). 
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Data Analysis 

Transcription and extraction. An independent, professional transcriber completed cdl 
transcriptions from the audio tapes. Following transcription, the first author used a method 
described by Butterfield el cd. (2009) to extract distinct CIs. To ensure that the CIs were 
extracted accurately (i.e., that the transcription was divided into unique CIs correctly), an 
independent researcher with qualitative research experience was provided a random sample of 
25% of interviews to independently extract CIs for intercoder agreement. Initial results 
indicated 96% agreement, which upon a consensus discussion was reused to 100%. 

Descriptive validity. Multiple methods were used to verify the accuracy of data collection 
according to procedures from Bedi el cd. (2005) and Butterfield el cd. (2005). First, to serve as a 
fidelity check for the interviewing process, cm expert in the field of CIT methodology analyzed 
the transcripts from the first two interviews to confirm adherence to the protocol and lack of 
leading questions. Second, to ensure saturation (the point at which which interviews ceased to 
generate new CIs or categories; Flanagan, 1954), the first author tabulated CIs after every three 
interviews to monitor the number of new categories. Saturation was achieved after the first five 
inten’iews. Further, at the end of each interview, the CIs manually recorded by the interviewer 
were read back to the interviewee in their own words, to cdlow them the opportunity to add, 
modify, or clarify a Cl (first level member checking; Brantlinger et cd., 2005). 
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Coding categories. Initial coding entailed an examination of the similarities and 
differences among helping CIs ( Butterfield et al., 2009) using open coding, in which data are 
segmented into categories of information best representing the “persistent ideas ” (Creswell, 
2007). The first author periodically analyzed CIs to determine patterns, categories, and 
commonalities, and then form preliminary categories through inductive clustering. CIs 
containing similar words, phrases and sentences formed emergent categories. As suggested by 
Butterfield and colleagues (2009), CIs were coded in sets, consisting of three randomly chosen 
interx’iews. Following this first extraction, the next sets of interviews were analyzed. 

Credibility and Trustworthiness of Data and Interpretations 
Although the process of coding data into distinctive thematic units is inherently 
subjective, five steps were taken to assess and secure the trustworthiness of the current findings 
(as recommended for CIT research by Butterfield et al., 2005). These steps closely align with 
the credibility checks and quality indicators for interview studies and qualitative data analysis 
outlined by Brantlinger and colleagues (2005) for special education research. First, expert 
feedback on the appropriateness and utility of the categorical titles and operational definitions 
was obtained from the second author, who was blinded to the coding process, through a process 
known as peer debriefing (Brantlinger et al., 2005). Second, results were compared to categories 
in the existing literature (Brantlinger et al., 2005; Butterfield et al., 2009). The majority of 
elements previously articulated in the literature review were reflected in the results obtained. The 
third verification step involved calculating intercoder agreement for sorting CIs into categories 
(Anderson & Nilsson, 1964). An independent rater, a published researcher in qualitative 
research, sorted a randomly selected 25% of CIs into the 13 categories, using a codebook with 
the category titles and a brief description of each category. The intercoder agreement was 87%, 
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which is considered to be strong according to Anderson and Nilesson (1964). The fourth 
accountability procedure, a second level member check (Brantlinger et al., 2005), was conducted 
to examine the interpretive validity of the Cl extractions and formulation of the categories. 
Member checks establish the trustworthiness of the researcher’s interpretation according to the 
intended meaning of the participant (Bedi et al., 2005). The first author provided participants 
with definitions of all of the final categories and asked them whether the categories captured 
their experiences. Overall, 73% of the participants completed this member check. Of these 
participants, all but one indicated that all of the categories resonated with their experiences (this 
participant suggested a different name for one category, which we adopted). The final step to 
assess trustworthiness was to set a minimum participation rate in a given category. Flanagan 
(1954) asserted that a category was valid only if “significant frequencies” of CIs are reported 
under that category. For this study, categories were deemed valid only if at least 25% of the 
participants reported a critical incident in that category (Butterfield et al., 2009). 

Positionality 

Qualitative research requires examining what personal perspectives may influence the 
research and data analysis process (Brantlinger et al., 2005). The first author, the primary 
researcher, is an educator with special education teaching experience with a strong belief in data- 
driven instruction but no previous experience in SWPBIS. The second and third authors are 
published researchers in the field of positive behavior support with an interest in identifying how 
SWPBIS can be sustained. Further, the second author provided training and technical assistance 
to the school district studied. The fourth author is a doctoral student with SWPBIS experience. 
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Findings 

The interviews generated 227 Critical Incidents (CIs) for analysis, far above the 
recommended minimum of 100 (Flanagan, 1954). Table 1 provides the 13 categories of CIs, 
including the number of helping and hindering incidents provided by participants, percentage of 
the total, and the number of participants providing the CIs. 

[Insert Table 1 here] 

Continuous Teaching 

Continuous Teaching, cited by 88% of participants, was perceived as a strong method to 
enhance SWPBIS sustainability. These CIs refer to consistent reteaching of expectations and 
social skills through classroom lessons, incidental teaching, assemblies, and presentations. 
According to participants, the experience of regularly seeing and hearing the expectations 
through various modes promotes a consistent, unified SWPBIS culture within schools. 
Participants expressed that the process of continuous teaching focused and operationalized the 
aim and daily activities of SWPBIS for school personnel, making the expectations “part of our 
culture. And so if you’re on staff you feel like you need to be a part of that culture.” For them, 
the explicit and ongoing communication helped their schools form a collective purpose. 

For our kids, common language and common understanding are the two biggest 
things that they need to have. They need to know that when I talk to them at that 
school about [expected] behavior that we’re all talking about the same thing. One 
of the major things that we continue to do. ..is a re-teaching of the matrix to our 
students. . . And quite frankly I think the big piece of it for me was that it was not 


only for kids. It was for staff. 
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Not surprisingly, a lack of continuous teaching was described as a hindering event. With 
a lack of review of core expectations and procedures, staff and student behavior can regress: 
“When we didn’t [reteach] ...we saw a greater increase in negative behavior... kids running in the 
hallways... teachers reacting.. .it became apparent that staff were contributing to the problem.” 

Positive Reinforcement 

Fourteen participants (82%) cited Positive Reinforcement, a diverse set of experiences, 
including using systems for positive reinforcement, focusing on prosocial behavior, and 
receiving positive feedback. It applies to both adults reinforcing students’ prosocial behavior and 
adults being reinforced (e.g., through improved student outcomes) for implementing SWPBIS: 
“It’s really giving the kids a double hit for every single [acknowledgment ticket] because the first 
hit is they got the acknowledgement but then they’re getting another opportunity to be reinforced 
by being able to trade it in for something at the store. . .[When students are] not getting positives 
at home. . .they need a second hit.” 

Participants reported that using the SWPBIS acknowledgment system occasioned student 
change, and observing that change occasioned adult implementation of SWPBIS. When asked to 
recall an event that helped sustainability, the vast majority noted detailed examples of students’ 
behavior improving upon receiving positive acknowledgment, and such improvement served as a 
stimulus for them. As noted by one administrator about her staff, “they believe that what we’re 
doing is actually making a difference because we see change in behavior.” Another participant 
noted, “if people find a very positive environment with children. . .they’re very respectful and so 
on. . .then you want to. ..It reinforces that what we’re doing is right and we don’t want to lose that, 
we want to keep going.” Sustainability was hindered when teams did not refresh aspects of the 
acknowledgement system, which made it less appealing to students and staff alike. 
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SWPBIS Team Effectiveness 

A total of 88% of participants discussed the importance of SWPBIS Team Effectiveness, 
the organizational structure within the school to support SWPBIS implementation. Effective 
teams met consistently and had a broad representation of staff members, with mechanisms for 
training staff and reporting back to the rest of the school faculty. When SWPBIS teams were 
well-formed, met regularly, and held a high profile in the school, they were identified as 
beneficial. As seen in the following excerpt, effective SWPBIS teams provided the 
organizational infrastructure to activate the SWPBIS framework and ensure follow-through for 
the whole school. 

We try to do [our monthly meetings] before our staff meetings so that we can 
relay the information to the staff members who can’t be there. . . We just don’t 
meet, we kind of revisit and we see what we talked about last time and where 
we’re going. I think it’s the piece like looking at what we’ve discussed last time 
and our success or, if there’s something that we didn’t get to and making sure that 
we’re making goals that are actually happening. 

Effective teams were able to maintain the conversations about SWPBIS at the school 
level and allowed people the space to voice concerns and share insights. Hindering CIs in this 
category identified a lack of organizational structure — such as no regular meetings, no defined 
roles, poor collaboration, or fragmented team member — as a barrier to sustainability. 

Staff Ownership 

Staff Ownership, highlighted by 76% of participants, refers to SWPBIS as a teacher- 
generated and owned initiative, as opposed to a top-down mandate imposed by administrators. 
This category centered on teacher buy-in and a high level of involvement in planning and 
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implementation. Participants often identified school champions, lead implementers with passion, 
credibility, and willingness to keep staff members moving forward with implementation. 

If it is given to you from the principal, then there is an automatic reaction, the 
question is whether I have to do it or I don't have to do it. I think that a lot of it 
came from the staff people, the people in the trenches [who] are dealing with 
some of the behaviors. It was teacher generated... coming from a teacher, and 
there is a reason for why you’re doing it, because it makes my job easier, because 
I can tell you as a colleague, hey I'm doing this and it's. . .staff driven. Not being 
told to do it. 

Despite turnover in staff and administrators, the grassroots nature of SWPBIS allowed it 
to continue even when champions and supportive administrators moved on. 

There was enough of us who were dedicated to make sure that the transition 
happened — that [SWPBIS] would happen no matter what — because our initial 
core group. . .knew what it was like before [SWPBIS]... to have people who, who 
saw the overall picture and although people have come in and left, you still had to 
have the people who had the vision of what it could be and maintain it. 

Adaptation 

Adaptation, noted by 71% of participants, refers to maintaining core elements while 
adapting daily practices to make them more efficient and effective. It also includes customizing 
SWPBIS practices to “fit” local school context (e.g., local cultural translation of expectations, 
changing practices to reflect student demographics) to keep practices relevant. As one 
participant reported, “we’re constantly having to re-evaluate both our systems and how those 
systems match with the student body and the behaviors of those students.” Specific adaptations 
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mentioned included changing acknowledgment systems, revising forms to make them more 
efficient, and using creative new ideas for teaching expectations. Practitioners explained how 
these changes both made SWPBIS more relevant for their students and families and recaptured 
enthusiasm from teachers and students: “We decide to try different things, not just doing the 
same things our committees have always done, because kids get tired of it, and the staff get tired 
of it, and then you lose the power of being inspired, finding things exciting.” 

Community of Practice 

In total, 65% of participants reported a Community of Practice as a helpful activity toward 
sustainability of SWPBIS practices. Events in this category describe networking and connections 
to peers implementing SWPBIS, as opposed to simply receiving training or coaching from an 
external expert. CIs provided discussion of annual provincial SWPBIS conferences and district 
networking sessions that engaged implementers in multi-level dialogue. Forums for presenting, 
sharing, and listening to others’ ideas, resources, and celebrations kept the conversation of 
SWPBIS alive: “It was more just having the opportunity to collaborate with other SWPBIS 
schools. I don’t think it was really a matter of what that collaboration looked like. I think it was 
the event of the collaboration that was valid and important.” 

Participants reported that networking served several important functions for sustainability. 
They referred to networking as a helpful buffer against the obstacle of staff turnover. Staff 
invited new teachers and principals to the meetings and left with strategic plans to focus 
SWPBIS. Exchanging information and ideas with other schools, sharing data, hearing how other 
school teams implement, and sharing concrete examples of practices (e.g., matrices, lesson 
plans) provided a vehicle to implement and refine existing practices. 
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Involving New Personnel 

Involving New Personnel, cited by 64% of participants, refers to bringing in new 
staff, with new ideas, energy, and a fresh perspective. Recruitment of new teachers onto 
the SWPBIS team builds capacity and increases immersion into the school SWPBIS 
culture, as well as enhanced buy-in from staff when champions rotate off of the team. 

Each school had specific plans for recruiting new teachers and administrators supportive 
of SWPBIS, such as job postings stating clearly that SWPBIS was a core school practice 
and experience with SWPBIS was a preferred hiring characteristic. Beyond hiring 
practices, participants noted the importance of having ongoing mechanisms in place to 
ensure that new staff understood SWPBIS. Mentor teachers were assigned to help new 
teachers learn the expectations and forms. These schools also encouraged all new staff to 
join the SWPBIS team. One participant recalled, “One of the staff members that came up 
to me when I first came to the school gave me a binder and explained everything in the 
binder even before I was full time.” Joining the SWPBIS team was seen as a way to 
introduce new teachers to the positive school culture and help build their teaching skills. 

First year teachers, they’re overwhelmed, but we kept saying, you know, we’d 
love to see you at a meeting, you know, it would really help out. And so when 
they go there, it just automatically includes, “you need to start understanding how 
it really works,” and they got involved. 

This process of bringing new members on to the teams and cycling veterans off had the 
additional beneficial effect of refreshing aspects of the SWPBIS systems as well. The fresh 
perspectives were seen as important to keeping the practices novel and interesting. As one 
participant noted, “over time ...new people who had fresh lenses brought in new ideas.” 
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Use of Data 

Use of Data, provided by 65% of participants, relates to having observable and 
measureable information to track patterns of both SWPBIS implementation and student 
outcomes. Data facilitate communication and accountability between staff, the school team, the 
school board, and parents: “Data speaks to the staff, ‘Look what we’ve accomplished; look 
where we need to go.’ It keeps it impersonal and really clarifies decision making for the future 
and how to celebrate the present and past.” 

Participants noted data collection as supporting high levels of implementation fidelity. 
Collecting and using data required teachers to focus on critical SWPBIS activities and guiding 
principles. It also provided opportunities to develop strategies and evaluate their effectiveness. 
The fact that at every meeting we’d pull the data out, and we look at it, and we 
see as the committee where the problem areas, for instance, the hallway, you 
know, if there’s an increase, okay, what’s contributing to that, how can we make 
this, the [office discipline referrals] go down? Or what do we need to do to make 
sure we’re writing [office discipline referrals] to change behavior? 

Access to External Expertise 

A total of 59% of participants highlighted the importance of Access to External Expertise, 
contact with a recognized researcher, consultant, or trained coach with outside information and 
tools. When systems become too narrowly focused or stagnant, the voice of an outside 
perspective can keep the system moving forward. As a participant noted, “it’s good to have 
outside feedback, someone from the outside look in. . .because our vision had become narrow, we 
get too much into ourselves.” Those interviewed cited the importance of involvement from 
coaches who were specially trained to provide SWPBIS implementation support. External 
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coaches (i.e., district staff who assist school teams in implementing SWPBIS) helped staff 
evaluate and troubleshoot daily practices. Coaches also connected district and school 
administrators and school staff so that they were building systemic district capacity together. 

Maintaining Priority 

Maintaining Priority, cited by 53% of participants, refers to SWPBIS being in the 
spotlight, having a high profile within the district, or being provincially endorsed. Participants 
emphasized the importance of prioritizing SWPBIS through written policy. Participants noted 
that maintaining SWPBIS as a priority provided validation and affected daily practice, funding, 
professional development selection, and accountability, which in turn affected sustainability. 

If the school goal was around social responsibility then it formalized the goal 
setting, the establishing objectives, collecting data, and reporting on it. It became 
part of the curriculum component... [just like] reading, writing, and 
spelling. ..Before that, it was kind of invisible. Having social responsibility in the 
provincial curriculum clearly establishes it within the whole educational 
framework. It was a critical event in validating schools for doing what they were 
doing. ..and gave them a context... it’s part of what we should be doing at schools. 
Participants also expressed that having SWPBIS written into the district action plan and 
goals sets it as a strong district priority, which ensures that it is viewed as important by schools. 
One district administrator reported, “if it is not in that document, it can easily be dropped.” 

School Administrator Involvement 

School Administrator Involvement, cited by 76% of participants, refers to the critical role 
of principals as agents who can either help or hinder sustainability. Participants described the 
value of administrators with a clear understanding of, and active support for SWPBIS. 
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They ’re the leader of the school, and that ’s a really important person to be at the front, 
spearheading and taking it seriously ...They ’re the ones who are going to be allowing 
programs and celebrations and assemblies and encouraging cdl those behaviors, and 
they ’re the ones who are going to find the time for [additional support and data 
collection] . So if they don ’t believe in it, they ’re not going to take the time doing it. 

Participants also noted CIs in which the principal ’s ability to listen and respect what has 
been done was critical to durability of practices. Teachers expressed that SWPBIS becomes a 
priority when the principal is actively involved in ground level implementation, such as 
consistently using and promoting SWPBIS language themselves, going into the classroom to 
directly teach expectations, and giving presentations to staff and community agencies. 

Staff Turnover 

Staff Turnover, noted by 47% of participants, refers to the hindering event of staff 
mobility. Staff turnover can diminish staff knowledge and skills in daily SWPBIS practices 
(e.g., continuous teaching, data-based decision making) and can reduce staff commitment and 
consistency. For example, “we’ve had so much staff turnover in the last two years. . .and 
[SWPBIS] hasn’t been explained, and we just kind of assume that people are on board.” 

Conflict in Personal Beliefs/Mistaken Beliefs 

Conflict in Personal Beliefs/Mistaken Beliefs, noted by 82% of participants, regarded two 
types of conflicts. First, they discussed the notions of different personal philosophies, values, and 
beliefs, which led to inconsistent implementation. For example, divergent beliefs in the areas of 
equity, social behavior norms, and rewards and consequences, were identified as barriers that 
often led to lack of engagement and poor implementation of practices. The following excerpt is 
a typical example of CIs cited under this category: “If they have a different philosophy and it 
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doesn’t fit here and they’ll just do their own thing and that hinders the process of sustainability.” 
From one administrator’s perspective, the belief that teachers should focus on academics while 
administration deals with behavior can also be a barrier: “they just want to teach their reading, 
writing, and arithmetic... I think we have that feeling in our building-administration should do 
something... aren’t getting it that it is a team approach.” A second aspect of this category regards 
mistaken beliefs about SWPBIS. One misconception was that writing office discipline referrals 
is in itself punitive, not realizing that collecting this type of information could help students in 
the long term and allow teachers to prevent challenges. Another misconception was that 
SWPBIS is only the use of external rewards: “. . .educators feeling that it’s only about 
[acknowledgment tickets], tangible reinforcements, and not understanding it’s something way 
bigger.” 

Discussion 

This study explored the perspectives of school and district personnel regarding events 
that affect Tier I SWPBIS sustainability. The CIT methodology, through richly detailed 
description, provides a comprehensive picture of sustainability as it unfolds in the real world. 
Data were generated from detailed interviews with 17 participants who were asked to report 
important events (i.e., specific behaviors or observable happenings) that helped or hindered the 
sustainability of SWPBIS. Examination of these data revealed thirteen categories of Critical 
Incidents (CIs). These categories are best understood within the context of the existing literature 
base, including confirming previous findings and elaborations or unique contributions. 
Confirmation of Existing Research 

Many of the findings of this study are generally consistent with existing conceptual 
models. Specifically, the findings provide additional evidence that staff buy-in, active 
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administrator support, external expertise, teaming, and the use of data are critical variables in 
sustaining school-based practices (Adelman & Taylor, 2003; Bambara et al., 2009; Coffey & 
Homer, 2012; Kincaid et al., 2007; McIntosh et al., 2013; McIntosh et al., 2014; Santangelo, 
2009). The CIs in these categories show the extreme importance of attending to these principles 
and incorporating them into sustainability action planning. 

Unique Contributions or Elaborations 

Because of CIT's pragmatic focus on specific events, findings often provide clear 
implications for practice, including strategies to address common and less understood issues. 
Some strategies appear to be unique to the literature, which can at times be more theoretical and 
lacking in explicit steps (McIntosh et al., 2010). Often, participants remarked that events in one 
category were synergistic, increasing the strengths of other categories as well. 

Continuous teaching for continuous regeneration. One cross-category finding was the 
notion that continuous teaching of expectations and prosocial behavior led to continuous 
regeneration of the practice. Participants strongly perceived continuous teaching as helpful in 
reactivating key SWPBIS components. In the CIs cited, it was evident that continuous teaching 
served as a reminder to both staff and students of the guiding principles of SWPBIS. Many staff 
members recalled that these conspicuous elements of SWPBIS were a regular reminder to use 
SWPBIS practices. Although educators sometimes consciously abandon programs, they may 
also slowly neglect practices over time, leading to their demise (Klingner, Vaughn, Hughes, & 
Arguelles, 1999). 

Similar to evolutionary principles, practice sustainability depends on the contextual fit of 
teaching strategies with the ever-changing environmental conditions. When core elements are 
adapted to address current challenges identified in the school and linked to important outcomes, 
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continued practice is more likely (Han & Weiss, 2005). These ideas are consistent with the 
finding that flexible initiatives that can be continuously adapted to fit with goals of the 
community are more sustainable (Pluye et al., 2005). 

Positive reinforcement. Another finding that has been seen in some research but was 
strongly highlighted here was positive reinforcement as a key mechanism for sustained 
implementation. In combination, the categories of Positive Reinforcement and Use of Data show 
the importance of self-sustaining feedback loops that highlight practice effectiveness and 
motivate future efforts (Han & Weiss, 2005). Participants consistently identified the principle of 
positive reinforcement as fundamental to sustainability. They reported their personal 
experiences of observing positive student outcomes (e.g., improved student behavior, 
achievement, and school climate) as reinforcing their sustained use of SWPBIS. Several studies 
have also recognized this principle’s importance to sustainability (Baker et al., 2004; Pluye et al., 
2005). 

Critical Incidents from these categories suggest the importance of implementers at the 
district and school level using data to highlight the connection between implementation and 
student success, strengthening the reinforcing properties of continued use. When staff 
experienced the positive benefits of data collection, their perceptions of the activity improved, as 
has been shown in other studies (Rogers, 2003). Data systems allowed SWPBIS teams to 
develop a sense of professional autonomy and become more self-sufficient in directing their own 
priorities and resources, further increasing positive reinforcement for using data. 

Importance of networking. Accessible networks for exchanging ideas, engaging in 
problem solving, and sharing successes enable ongoing professional development of technical 
skills and a common knowledge base (Hieneman & Dunlap, 2001). Participants reported that 
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their connection to a community of practice in their district provided not only ongoing 
information and training, but also a sense of a common mission, which strengthened and 
renewed their commitment (Klingner et al., 1999). According to Baker and colleagues (2004), 
teachers with experience in the practice reported that their greatest concern was for increased 
collaboration among professionals, whereas the concerns of teachers new to the practice were 
primarily related to basic implementation procedures. 

Bringing new staff into the school culture. According to participants and the literature, 
staff turnover seriously threatens sustainability (Adelman & Taylor, 2003). In response, 
participants noted their deliberate, focused efforts and strategic plans to involve new staff 
members into their SWPBIS community of practice. These strategies included (a) requiring or 
strongly encouraging new staff to join the school SWPBIS team, (b) pairing them with SWPBIS 
team veterans for mentoring, (c) encouraging participation in networking sessions and 
conferences, and (d) changing job descriptions and hiring procedures to require experience or 
support for SWPBIS, thus institutionalizing the practice into policy (Fixsen et al., 2005). 

Involving new staff was also seen as a specific strategy to enhance other categories of 
sustainability. For example, participants noted that Staff Ownership was strongest when 
SWPBIS systems were staff-embraced, grassroots systems change initiatives. This perception 
was enhanced in part by bringing new staff onto the team and into the fold immediately. This 
strategy also helped to mitigate the most commonly reported hindering category, Conflict of 
Personal Beliefs/Mistaken Beliefs. Early immersion of incoming staff in the school’s SWPBIS 
culture was reported to increase acceptability and dispel misrules regarding the practice. 

School administrator support as barrier. In keeping with the existing literature 
(Bambara et al., 2009; Kincaid et al., 2007; Santangelo, 2009), school administrator support was 



Running head: SUSTAINING SCHOOL- WIDE PBIS 


25 


identified as important by a majority of participants. However, one interesting finding was the 
proportion of helping to hindering events. Participants noted more hindering than helping CIs in 
this category. This pattern may indicate some benefits of a supportive principal, but even 
stronger drawbacks of an unsupportive principal (Bambara et al., 2009; Pinkelman et al., 2014). 
These findings may explain seemingly contradictory findings that school personnel perceive 
school administrators as the most important factor in promoting sustainability (McIntosh et al., 
2014), whereas empirical studies from the same sample indicate administrator support as less 
influential than effective teaming and use of data for decision making (McIntosh et al., 2013). As 
a result, school and district teams may want to build action plans not only for recruiting 
supportive administrators, but also for ensuring sustainability through administrative turnover 
(Strickland-Cohen, McIntosh, & Horner, 2014), as well as enhancing support from skeptical 
administrators (McIntosh et al., in press). 

Implications 

The events documented in this study illustrate that maintaining interest and motivation for 
a practice among a sea of competing initiatives requires foresight and creativity. Site-based 
plans can include procedures that support sustainability, such as presenting outcomes to other 
stakeholders (e.g., school boards, parent organizations). School administrators can take a more 
active role in supporting formal organizational structures, blending initiatives, recruitment, 
monitoring ongoing staff commitment, facilitating networking and creating an ethos of collective 
efficacy. Their active involvement on the team, modeling and re-teaching of core features, 
setting school priorities, and emphasizing data use can positively affect preservation of practices. 
School staff can be supported through networking meetings, SWPBIS team activities, continuous 
feedback, and modeling of continuous teaching of core components to foster the establishment of 
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a common language. Adelman and Taylor (2003) suggested that institutionalization of practices 
occurs when school personnel actively take on long-term ownership and have a blueprint to 
offset events that threaten implementation fidelity. This study affirms the need for specific 
strategies to enhance support and counter staff turnover, such as bringing in new personnel as 
early as possible to improve implementation, enhance continuity, and bring fresh ideas. 

Limitations and Future Directions 

The findings from this exploratory study should be viewed with specific methodological 
limitations. First, the study relied upon retrospective self-report, which may be selectively or 
imperfectly recalled. In addition, beneficial and hindering events were subjectively recalled as 
personal reconstructions, which may have been biased by availability heuristics, attributions, and 
cultural artifacts (Gilovich, Keltner & Nisbett, 2006). However, CIT includes a mechanism 
designed to protect against this weakness — its reliance on multiple sources of information allows 
researchers to identify inconsistencies in accounts and “minimize the effects of a particular faulty 
recall on the part of an individual participant” (Kain, 2004, p. 79). In this study, we analyzed 
categories across participants, eliminated CIs without sufficient representation, and performed 
member checks at different points in time to reduce potential inconsistencies. Nevertheless, these 
categories might not be replicated in large-scale empirical studies. To address these issues, future 
studies could include direct behavior observations and analysis of permanent records (e.g., 
SWPBIS team meeting agendas, coaching logs) to corroborate events cited by participants. 

Finally, these results may be most pertinent to elementary schools where Tier I SWPBIS 
has been sustained. We cannot assume that non- sustaining schools, middle or high schools, or 
schools implementing SWPBIS at Tiers II and III would identify the same enablers and barriers. 
Additional research could include a more diverse range of both sustaining and non- sustaining 
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schools implementing all tiers. Future replications, with more individual respondents across 
roles, will be needed to assess the validity of these findings beyond this particular sample. 
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Table 1 


Critical Incidents by category and contributing participants. 


Category 

Helping 
Incidents 
(% of total) 

Hindering 
Incidents 
(% of total) 

Contributing 
Participants 
(% of total) 

Continuous Teaching 

21 (12.73%) 

3 (4.84%) 

15 (88.24%) 

Positive Reinforcement 
SWPBIS Team 

14 (8.48%) 

2 (3.23%) 

14 (82.35%) 

Effectiveness 

14 (8.48%) 

5 (8.06%) 

15 (88.24%) 

Staff Ownership 

22 (13.33%) 


13 (76.47%) 

Adaptation 

20(12.12%) 

1 (1.61%) 

12 (70.59%) 

Community of Practice 

17 (10.30%) 


11 (64.71%) 

Involving New Personnel 

13 (7.88%) 


11 (64.71%) 

Use of Data 

13 (7.88%) 

3 (4.83%) 

11 (64.71%) 

Access to External Expertise 

16 (9.70%) 


10 (58.82%) 

Maintaining Priority 
School Administrator 

8 (4.85%) 

9 (14.52%) 

9 (52.94%) 

Involvement 

7 (4.24%) 

12 (19.35%) 

13 (76.47%) 

Staff Turnover 


10(16.13%) 

8 (47.06%) 

Conflict of Personal Beliefs 


17 (27.42%) 

14 (82.35%) 



